基于DDPG算法的离网型可再生能源大规模制氢系统优化调度

doi:10.3969/j.issn.2097-0706.2024.06.005

摘要/Abstract

摘要：

为提高可再生能源消纳能力，减少整流和并网等设备的投资成本，降低电解水制氢系统成本，实现可再生能源大规模制氢，构建了一个孤岛型可再生能源大规模制氢系统。该系统通过智慧能量管理，实现了提高系统经济性与安全性的目标。首先建立可再生能源大规模制氢系统的仿真模型，制定控制策略;其次，提出一种基于深度确定性策略梯度（DDPG）的能量优化调度策略。通过大量长期的训练，使用DDPG算法得到的智能体能够实现智能化的动态能量优化调度。将该策略与深度Q网络、粒子群优化和传统控制方法在经济性和安全性方面进行比较，结果表明DDPG算法在能量优化管理中可实现更高的经济收益，更好地利用可再生资源，并确保系统的安全运行。

关键词: 可再生能源, 大规模制氢, 离网型, 深度确定性策略梯度, 优化调度

Abstract:

To improve the renewable energy consumption， reduce the investment on rectifiers and grid connection equipment， cut down the cost of water electrolysis for hydrogen production through powering hydrogen production by renewable energy， an islanded renewable energy large-scale hydrogen production system is constructed. An intelligent energy management platform can improve the economy and safety of the system. Firstly， a simulation model of the renewable energy large-scale hydrogen production system is established and its control strategy is formulated. Secondly， an energy optimization scheduling strategy based on deep deterministic policy gradient （DDPG） algorithm is proposed. Through long-term trainings， the agent obtained from the DDPG algorithm can achieve intelligent dynamic optimized scheduling on energy. Comparing the performances of the proposed strategy with deep Q network （DQN）， Particle Swarm Optimization （PSO） and traditional control methods in terms of economy and safety， it is shown that applying the DDPG algorithm in energy optimization and management can get higher economic returns and utilization rates of renewable resources， and ensure the safe operation of the system.

Key words: renewable energy, large-scale hydrogen production, off-grid, deep deterministic policy gradient, optimized scheduling

中图分类号:

TK91：TM715

郑庆明, 井延伟, 梁涛, 柴露露, 吕梁年. 基于DDPG算法的离网型可再生能源大规模制氢系统优化调度[J]. 综合智慧能源, 2024, 46(6): 35-43.

ZHENG Qingming, JING Yanwei, LIANG Tao, CHAI Lulu, LYU Liangnian. Optimized scheduling on large-scale hydrogen production system for off-grid renewable energy based on DDPG algorithm[J]. Integrated Intelligent Energy, 2024, 46(6): 35-43.

图/表 19

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

图15

表1

表2

表3

表4

参考文献 17

[1]	ZHANG Q, CHEN W M, LING W. Policy optimization of hydrogen energy industry considering government policy preference in China[J]. Sustainable Production and Consumption, 2022, 33: 890-902.
[2]	韩世旺, 赵颖, 张兴宇, 等. 面向碳中和的新型电力系统氢储能调峰技术研究[J]. 综合智慧能源, 2022, 44(9):20-26. doi: 10.3969/j.issn.2097-0706.2022.09.003
	HAN Shiwang, ZHAO Ying, ZHANG Xingyu, et al. Researches on hydrogen storage peak‑shaving technology for new power systems to achieve carbon neutrality[J]. Integrated Intelligent Energy, 2022, 44(9):20-26. doi: 10.3969/j.issn.2097-0706.2022.09.003
[3]	HAO W, ZHANG S X, LI X X, et al. A multivariate coupled economic model study on hydrogen production by renewable energy combined with off‑peak electricity[J]. International Journal of Hydrogen Energy, 2022, 47(58):24481-24492.
[4]	葛磊蛟, 崔庆雪, 李明玮, 等. 风光波动性电源电解水制氢技术综述[J]. 综合智慧能源, 2022, 44(5):1-14. doi: 10.3969/j.issn.2097-0706.2022.05.001
	GE Leijiao, CUI Qingxue, LI Mingwei, et al. Review on water electrolysis for hydrogen production powered by fluctuating wind power and PV[J]. Integrated Intelligent Energy, 2022, 44(5):1-14. doi: 10.3969/j.issn.2097-0706.2022.05.001
[5]	KAFETZIS A, ZIOGOU C, Panopoulos K D, et al. Energy management strategies based on hybrid automata for islanded microgrids with renewable sources, batteries and hydrogen[J]. Renewable and Sustainable Energy Reviews, 2020, 134:110118.
[6]	MOHAMMED A, GHAITHAN A M, AL-HANBALI A, et al. A multi‑objective optimization model based on mixed integer linear programming for sizing a hybrid PV‑hydrogen storage system[J]. International Journal of Hydrogen Energy, 2023, 48(26):9748-9761.
[7]	FANG R M. Multi‑objective optimized operation of integrated energy system with hydrogen storage[J]. International Journal of Hydrogen Energy, 2019, 44(56):29409-29417.
[8]	CHENG T, ZHU X Q, GU X Y, et al. Stochastic energy management and scheduling of microgrids in correlated environment: A deep learning‑oriented approach[J]. Sustainable Cities and Society, 2021, 69:102856.
[9]	HARROLD D, CAO J, FAN Z. Renewable energy integration and microgrid energy trading using multi‑agent deep rein‑ forcement learning[J]. Applied Energy, 2022, 318:119151.
[10]	刘俊峰, 陈剑龙, 王晓生, 等. 基于深度强化学习的微能源网能量管理与优化策略研究[J]. 电网技术, 2020, 44(10):3794-3803.
	LIU Junfeng, CHEN Jianlong, WANG Xiaosheng, et al. Energy management and optimization of multi‑energy grid based on deep reinforcement learning[J]. Power System Technology, 2020, 44(10):3794-3803.
[11]	张自东, 邱才明, 张东霞, 等. 基于深度强化学习的微电网复合储能协调控制方法[J]. 电网技术, 2019, 43(6):1914-1921.
	ZHANG Zidong, QIU Caiming, ZHANG Dongxia, et al. A coordinated control method for hybrid energy storage system in microgrid based on deep reinforcement learning[J]. Power System Technology, 2019, 43(6):1914-1921.
[12]	余宏晖. 基于深度强化学习的微电网能量优化管理研究[D]. 广州: 华南理工大学, 2021.
	YU Honghui. Research on energy optimization management of microgrid based on deep reinforcement learning[D]. Guangzhou: South China University of Technology, 2021.
[13]	孙惠娟, 段伟男, 陈俐, 等. 基于乐观行动-评判深度强化学习的含氢综合能源系统低碳经济调度[J]. 电网技术, 2024, 48(5):1873-1883.
	SUN Huijuan, DUAN Weinan, CHEN Li, et al. Low‑carbon economic scheduling of hydrogen integrated energy system based on optimistic actor-critic deep reinforcement learning[J]. Power System Technology, 2024, 48(5):1873-1883.
[14]	梁涛, 孙博峰, 谭建鑫, 等. 基于深度强化学习算法的风光互补可再生能源制氢系统调度方案[J]. 高电压技术, 2023, 49(6):2264-2275.
	LIANG Tao, SUN Bofeng, Tan Jianxin, et al. Scheduling scheme of wind-solar complementary renewable energy hydrogen production system based on deep reinforcement learning[J]. High Voltage Engineering, 2023, 49(6):2264-2275.
[15]	彭刘阳, 孙元章, 徐箭, 等. 基于深度强化学习的自适应不确定性经济调度[J]. 电力系统自动化, 2020, 44(9):33-42.
	PENG Liuyang, SUN Yuanzhang, XU Jian, et al. Self‑adaptive uncertainty economic dispatch based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2020, 44(9):33-42.
[16]	王桂兰, 张海晓, 刘宏, 等. 基于近端策略优化算法含碳捕集的综合能源系统低碳经济调度[J]. 计算机应用研究, 2024, 41(5):1508-1514.
	WANG Guilan, ZHANG Haixiao, LIU Hong, et al. Low carbon economic scheduling of integrated energy systems based on proximal policy optimization algorithm with carbon capture[J]. Application Research of Computers, 2024, 41(5):1508-1514.
[17]	江昌旭, 刘晨曦, 林铮, 等. 基于深度强化学习的电力系统暂态稳定控制策略研究综述[J]. 高电压技术, 2023, 49(12):5171-5186.
	JIANG Changxu, LIU Chenxi, LIN Zheng, et al. Review of power system transient stability control strategies based on deep reinforcement learning[J]. High Voltage Engineering, 2023, 49(12):5171-5186.

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	314.30	287.77	252.35	236.65
损失费用/美元	599.64	559.64	723.03	516.28
卖氢收益/美元	4 000	4 000	4 000	4 000
总利润/美元	3 086.06	3 152.59	3 024.62	3 247.07
丢弃功率/kW	2 998.18	2 798.18	3 615.17	2 581.38
深度过充时间/h	9	3	0	0
深度过放时间/h	0	3	0	0

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	216.72	203.64	181.15	181.55
损失费用/美元	0	26.37	231.16	0
卖氢收益/美元	3 000	3 000	3 000	3 000
总利润/美元	2 783.28	2 769.99	2 587.69	2 918.45
丢弃功率/kW	0	131.84	1 155.78	0
深度过充时间/h	0	0	0	0
深度过放时间/h	0	0	0	0

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	204.38	204.38	175.2	182.92
损失费用/美元	234.44	234.44	318.17	62.00
卖氢收益/美元	3 000	3 000	3 000	3 000
总利润/美元	2 561.18	2 561.18	2 506.63	2 755.08
丢弃功率/kW	1 172.22	1 172.22	1 590.83	310.00
深度过充时间/h	16	16	0	0
深度过放时间/h	0	0	0	0

参数	控制策略	PSO	DQN	DDPG
运行成本/美元	296.31	296.31	261.64	297.01
损失费用/美元	400.01	400.01	416.51	136.02
卖氢收益/美元	4 000	4 000	4 000	4 000
总利润/美元	3 303.68	3 303.68	3 321.85	3 566.97
丢弃功率/kW	2 000.07	2 000.07	2 082.57	680.09
深度过充时间/h	6	6	0	0
深度过放时间/h	1	1	0	0